Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenes

The success of deep learning in computer vision is based on availability of large annotated datasets. To lower the need for hand labeled images, virtually rendered 3D worlds have recently gained popularity. Creating realistic 3D content is challenging on its own and requires significant human effort. In this work, we propose an alternative paradigm which combines real and synthetic data for learning semantic instance segmentation and object detection models. Exploiting the fact that not all aspects of the scene are equally important for this task, we propose to augment real-world imagery with virtual objects of the target category. Capturing real-world images at large scale is easy and cheap, and directly provides real background appearances without the need for creating complex 3D models of the environment. We present an efficient procedure to augment real images with virtual objects. This allows us to create realistic composite images which exhibit both realistic background appearance and a large number of complex object arrangements. In contrast to modeling complete 3D environments, our augmentation approach requires only a few user interactions in combination with 3D shapes of the target object. Through extensive experimentation, we conclude the right set of parameters to produce augmented data which can maximally enhance the performance of instance segmentation models. Further, we demonstrate the utility of our approach on training standard deep models for semantic instance segmentation and object detection of cars in outdoor driving scenes. We test the models trained on our augmented data on the KITTI 2015 dataset, which we have annotated with pixel-accurate ground truth, and on Cityscapes dataset. Our experiments demonstrate that models trained on augmented imagery generalize better than those trained on synthetic data or models trained on limited amount of annotated real data

A Study of Lagrangean Decompositions and Dual Ascent Solvers for Graph Matching

We study the quadratic assignment problem, in computer vision also known as graph matching. Two leading solvers for this problem optimize the Lagrange decomposition duals with sub-gradient and dual ascent (also known as message passing) updates. We explore s direction further and propose several additional Lagrangean relaxations of the graph matching problem along with corresponding algorithms, which are all based on a common dual ascent framework. Our extensive empirical evaluation gives several theoretical insights and suggests a new state-of-the-art any-time solver for the considered problem. Our improvement over state-of-the-art is particularly visible on a new dataset with large-scale sparse problem instances containing more than 500 graph nodes each.Comment: Added acknowledgment

Orthodontic treatment needs in the western region of Saudi Arabia: a research report

BACKGROUND: Evaluation of self perceived and actual need for orthodontic treatment helps in planning orthodontic services and estimating the required resources and man power. In the present study, the perceptive need as evaluated by patients and the actual need to orthodontic treatment, as assessed by orthodontists, were evaluated at two types of dental practices in the city of Jeddah using the Index of Orthodontic Treatment Need (IOTN). METHODS: A consecutive sample of 743 adults seeking orthodontic treatment at two different types of dental practices in Jeddah; King Abdulaziz University, Faculty of Dentistry (KAAU) (Free treatment) and two private dental polyclinics (PDP) (Paid treatment), was examined for orthodontic treatment need using the dental health component (DHC) of the IOTN. The self-perceived need for orthodontic treatment was also determined using the aesthetic component (AC) of the IOTN. The IOTN score and the incidence of each variable were calculated statistically. AC and DHC categories were compared using the Chi-Square and a correlation between them was assessed using Spearman's correlation test. AC and DHC were also compared between the two types of dental practices using the Chi-Square. RESULTS: The results revealed that among the 743 patients studied, 60.6% expressed no or slight need for treatment, 23.3% expressed moderate to borderline need and only16.1% thought they needed orthodontic treatment. Comparing these estimates to professional judgments, only 15.2% conformed to little or no need for treatment, 13.2% were assessed as in borderline need and 71.6% were assessed as in need for treatment (p < 0.001). Spearman's correlation test proved no correlation (r = -.045) between the two components. Comparing the AC and the DHC between the KAAU group and PDP group showed significant differences between the two groups (p < 0.001). CONCLUSION: Patient's perception to orthodontic treatment does not always correlate with professional assessment. The IOTN is a valid screening tool that should be used in orthodontic clinics for better services especially, in health centers that provide free treatment

Normative and self-perceived orthodontic treatment need of a Peruvian university population

BACKGROUND: Previous studies on orthodontic treatment need in young adults have shown that up to 50% had malocclusions that needed orthodontic treatment. The aims of this study were to assess the normative and self-perceived need for orthodontic treatment using the Index of Orthodontic Treatment Need (IOTN) and to determine if the treatment need levels were influenced by sex, age and socio-economic status (SES) in a sample of Peruvian young adults. METHODS: 281 first-year students (157 male and 124 female students) with a mean age of 18.1 +/- 1.6 years were randomly selected and evaluated through the Dental Health Component (DHC) and Aesthetic Component (AC) of the IOTN. Structured interview and clinical examination were used to assess the students. Descriptive statistics and Chi-square tests were used for data analysis with statistical significance set at P < 0.05. RESULTS: An intra-examiner reliability of 0.89 was obtained (weighted Kappa). The percentage of students according to SES was 51.2%, 40.6% and 8.2% corresponding to low, medium and high SES respectively. The percentage of students with DHC grades 4–5 was 29.9% whereas the percentage of students with AC grades 8–10 was 1.8%. There were no significant differences in the distribution of normative and self-perceived orthodontic treatment need based on sex, age and SES comparisons. CONCLUSION: Normative orthodontic treatment need was not matched by a similar level of self-perceived treatment need in these young adults. Sex, age and SES were non-significant factors associated with levels of treatment need

Deep-learning based image synthesis with geometric models

Data­driven machine learning approaches have made computer vision solutions more robust and easily adaptable to various circumstances. However, they are often limited by their dependency on large datasets with accurate ground­truth annotations for training. In most scene understanding tasks, like instance segmentation and object detection, training data is often scarce since annotations can not be measured directly from the real world using sensors but instead have to be manually created by humans at large cost. Virtual scenes could offer a feasible alternative in these cases since the full access to the underlying scene geometry enables generating fast and accurate annotations. However, scene understanding models trained on rendered images often do not perform well on real test images due to the difference in appearance between the synthetic and real images. This thesis proposes several new methods for images synthesis with focusing on generating training images that could partially or totally replace real data for training deep-learning models. It first explore the use of augmented reality techniques for combining synthetic 3D objects and real scenes. This can greatly reduce the effort needed for generating diverse training scenes with accurate annotations. We study and compare the effect of various factors of image generation on the performance of the trained scene understanding models. To overcome the limitations of rendering engines, we next propose a novel geometric image synthesis approach that generates geometrically consistent and controllable images. The deep neural network learns to imitate the rendering process while at the same time optimizing for an explicit realism objective making the resulting images more suitable to train scene understanding models. Finally, to alleviate the need for rendered images, we introduce an unsupervised neural rendering model trained only using unpaired 3D models and real images of similar object class. This is achieved by learning the forward rendering and backward decomposition processes jointly. The results in this thesis indicate that deep-­learning based image synthesis models could be an efficient tool for generating realistic images and high­quality synthetic training data